Hey! 👋 This PR adds back support for Miniflare 2's Miniflare#get*() methods for accessing bindings from outside the Workers sandbox. It does this by implementing a proxy system, that makes HTTP requests to workerd on property access/function calls. Essentially the aim here is to support...

import { Miniflare } from "miniflare";
const mf = new Miniflare(...);
const ns = await mf.getDurableObjectNamespace("OBJECT");
const id = ns.idFromName("thing");
console.log(id.toString());
const stub = ns.get(id);
const res = await stub.fetch("http://placeholder/");
console.log(await res.text());

...without re-implementing any of the APIs in Node.js. Note the above snippet uses a mix of synchronous and asynchronous calls, and uses some native objects as arguments to other native functions.

Very cool projects such as https://github.com/leader22/cfw-bindings-wrangler-bridge and https://github.com/james-elicx/cf-bindings-proxy already exist and do similar things. This PR aims to improve on those solutions by:

• Supporting synchronous methods like those on DurableObjectNamespace
• Supporting passing native return values from one method as parameters of another
• Avoiding re-declaring the existing API (we're still doing this with types atm, but should be able to avoid that too)
• Avoiding buffering ReadableStream arguments and return values where possible
• Being general enough to avoid special casing APIs where possible

Note this solution relies on a persistent Durable Object that you probably wouldn't want to run in remote preview. The approaches taken by the projects above are likely more appropriate there.

High Level Implementation Overview

The proxy client is implemented in src/plugins/core/proxy/. The server is implemented as a Durable Object in src/workers/core/proxy.worker.ts.

Using a Durable Object allows us to share references containing I/O objects like streams across requests. The Durable Object contains a "heap", mapping addresses to references. The "heap" is initialised with globalThis and env.

The proxy client builds proxies for each object on the heap. Accessing a property on a proxy will result in a synchronous GET operation to the proxy server. If the property is not a method, the value will be serialised back to the client. Note this may involve putting more references on the "heap".

If the property is a method, the client will return a function from the access. Future accesses to the same property key will return the same function without the synchronous GET operation. Calling this function will serialise all the arguments, then perform a synchronous CALL operation on the target, and the return value will be serialised back to the client. If this function returns a Promise, it will be stored on the "heap", and a reference returned. An asynchronous GET operation will then be performed to resolve the Promise, and serialise the result. If a function returns a Promise once, all future calls will send asynchronous CALL operations instead, that resolve the Promise without an additional round trip.

If the function call had ReadableStream arguments, the first will be sent unbuffered after the rest of the arguments. All function calls with ReadableStream or Blob arguments are assumed to be asynchronous. This assumption is required as synchronous operations block the main thread and prevent chunks from ReadableStreams being read.

If the function call threw, or returned a Promise that rejected, the error will be serialised and re-thrown/rejected in the client. Note that the stack trace will be updated to reflect the calling location in the client, not the server.

To prevent unbounded growth of the "heap", all proxies are registered with a FinalizationRegistry. When the proxy is garbage collected, a FREE operation will remove the corresponding "heap" entry allowing it to be garbage collected on the server too.

When workerd is restarted with Miniflare#setOptions() or stopped with Miniflare#dispose(), all proxies are poisoned. Once a proxy is poisoned, it cannot be used, and must be recreated. Poisoned proxies are unregistered from the FinalizationRegistry too, as the addresses they point too will be invalid and shouldn't be freed again.